{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Advanced Plotting: Beyond matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright, so our prior charts were plotted using `matplotlib` which helps us see the data, but the charts don't look amazing and aren't interactive at all. What other options do we have? [seaborn](https://seaborn.pydata.org/index.html) is a wrapper the top of `matplotlib` that makes much prettier plots (with a ton of statistical plotting capabilities) and [plotly](https://plot.ly/) is a great cross-platform plotting liberary that we can easil set up. Below we'll use `seaborn` and `plotly` to make some pretty pictures. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's import some libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import datetime as dt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import datetime as dt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we don't have access to some of the data that we do within JPM, we'll have to generate some data to plot. In this case, we're going to generate an approximation of Russell2k implied volatility and HYG (iTraxx High Yield Bond ETF) prices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vols = [ 0.3, 0.06 ]\n",
    "dailyVols = vols / np.sqrt( 252 )\n",
    "corr = -0.4\n",
    "covars = [ \n",
    "    [ dailyVols[ 0 ] ** 2, dailyVols[ 0 ] * dailyVols[ 1 ] * corr ],\n",
    "    [ dailyVols[ 0 ] * dailyVols[ 1 ] * corr, dailyVols[ 1 ] ** 2 ]\n",
    "]\n",
    "randomSeries = np.random.multivariate_normal( ( 0.001, 0 ), covars, 500 ).T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "randomSeries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've got two return series, but we need to convert them to a time series for what they're meant to represent. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rtyVol = 0.2 * ( 1 + randomSeries[ 0 ] ).cumprod()\n",
    "hygPrice = 80 * ( 1 + randomSeries[ 1 ] ).cumprod()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if they make sense... Often the easiest way to do that is to plot them. Many of the plotting libraries set up to operate / plot a `DataFrame` natively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(np.array([rtyVol, hygPrice]).T, columns=[\"RTY.3m.Proxy.Implied.Vol\", \"HYG.spot\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So... problem #1, these two series not similar magnitudes. We need to plot these on difference axes and while we're at it let's make it look a little better. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.style.use('seaborn')\n",
    "df.plot(secondary_y=[\"HYG.spot\"], legend=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we flip around RTY implied vol, we can see this inverse relationship a bit better. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"RTY.3m.Proxy.Implied.Vol\"] = df[\"RTY.3m.Proxy.Implied.Vol\"] * -1\n",
    "df.plot(secondary_y=[\"HYG.spot\"], legend=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can look at the scatter on levels pretty easily with `matplotlib` using `matplotlib.pyplot.scatter`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter( df[ df.columns[ 0 ] ], df[ df.columns[ 1 ] ] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like there is a relationship there (there should be, we generated the series with a negative correlation). Let's explore that a bit. As we said before, `seaborn` comes with a bunch of good statistical tools. In fact, it has has quick and easy way to generate a regression plot with `sns.regplot`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots( sharex=True )\n",
    "sns.regplot( x=\"HYG.spot\", y=\"RTY.3m.Proxy.Implied.Vol\", data=df.diff(), ax=ax )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, the underlying regression data is not exposed in `seaborn`, so we will need to generate it ourselves using `scipy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import stats\n",
    "\n",
    "diff = df.diff().dropna()\n",
    "slope, intercept, rvalue, pvalue, stderr = stats.linregress(diff[\"HYG.spot\"], diff[\"RTY.3m.Proxy.Implied.Vol\"])\n",
    "print( \"R^2 = {r:.3f}\".format( r=rvalue ) )\n",
    "print( 'y = {m:.3f}x {sign} {b:.3f}'.format( m=slope, sign=\"+\" if intercept >= 0 else \"-\", b=abs(intercept) ) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if I want to interact with the plot.... zoom in, inspect the values, etc. This is where `plotly` shines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from plotly.offline import download_plotlyjs, init_notebook_mode, iplot\n",
    "import plotly.tools as tls\n",
    "init_notebook_mode(connected=True)\n",
    "\n",
    "# Here we can convert our matplotlib object to a plotly object\n",
    "plotlyFig = tls.mpl_to_plotly(fig)\n",
    "\n",
    "# Add annotation so you have the regression stats\n",
    "plotlyFig['layout']['annotations'] = [\n",
    "    dict(\n",
    "        x=18,\n",
    "        y=-2,\n",
    "        showarrow=False,\n",
    "        text='R^2 = {:.3f}'.format( rvalue )\n",
    "    ),\n",
    "    dict(\n",
    "        x=18,\n",
    "        y=-2.6,\n",
    "        showarrow=False,\n",
    "        text='y = {m:.3f}x {sign} {b:.3f}'.format( m=slope, sign=\"+\" if intercept >= 0 else \"-\", b=abs(intercept) )\n",
    "    )\n",
    "]\n",
    "iplot(plotlyFig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2d plots are cool but 3d plots.... Below is an example of plotting a vol surface from start to finish."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = [0.8023,0.814,0.8256,0.8372,0.8488,0.8605,0.8721,0.8837,0.8953,0.907,0.9186,0.9302,0.9419,0.9535,0.9651,0.9767,0.9884,1,1.0116,1.0233,1.0349,1.0465,1.0581,1.0698,1.0814,1.093,1.1047,1.1163,1.1279,1.1395,1.1512,1.1628,1.1744,1.186,1.1977,1.2093]\n",
    "i = [ dt.datetime(2019,8,2),  dt.datetime(2019,8,9),  dt.datetime(2019,8,16),  dt.datetime(2019,8,23),  dt.datetime(2019,8,30),  dt.datetime(2019,9,6),  dt.datetime(2019,9,20),  dt.datetime(2019,10,18),  dt.datetime(2019,11,15),  dt.datetime(2019,12,20),  dt.datetime(2019,12,31),  dt.datetime(2020,1,17),  dt.datetime(2020,3,20),  dt.datetime(2020,3,31),  dt.datetime(2020,6,19) ]\n",
    "d = [ [0.4244,0.4016,0.3796,0.3584,0.3381,0.3187,0.3002,0.2827,0.2662,0.2508,0.2363,0.2229,0.2105,0.1991,0.1888,0.1794,0.171,0.1636,0.157,0.1513,0.1465,0.1425,0.1393,0.1368,0.135,0.1338,0.1331,0.1329,0.133,0.1334,0.1341,0.135,0.136,0.1372,0.1384,0.1396],\n",
    "    [0.4006,0.3777,0.3556,0.3343,0.3139,0.2944,0.2759,0.2583,0.2418,0.2263,0.2118,0.1983,0.1859,0.1745,0.1641,0.1547,0.1463,0.1388,0.1322,0.1265,0.1216,0.1176,0.1143,0.1118,0.11,0.1088,0.1081,0.1078,0.108,0.1084,0.1092,0.1101,0.1112,0.1123,0.1136,0.1149],\n",
    "    [0.3431,0.3257,0.3089,0.2927,0.277,0.2621,0.2477,0.2341,0.2212,0.2089,0.1974,0.1866,0.1765,0.1671,0.1584,0.1504,0.1431,0.1364,0.1303,0.1248,0.1198,0.1154,0.1116,0.1083,0.1055,0.1032,0.1013,0.0998,0.0986,0.0977,0.0971,0.0966,0.0964,0.0963,0.0963,0.0965],\n",
    "    [0.3124,0.298,0.284,0.2705,0.2574,0.2449,0.2328,0.2213,0.2104,0.1999,0.1901,0.1807,0.1719,0.1637,0.156,0.1488,0.1421,0.136,0.1304,0.1253,0.1206,0.1165,0.1129,0.1098,0.1072,0.1049,0.1031,0.1015,0.1003,0.0994,0.0988,0.0983,0.098,0.0979,0.0978,0.0979],\n",
    "    [0.2955,0.283,0.2708,0.259,0.2475,0.2365,0.2259,0.2158,0.206,0.1968,0.1879,0.1795,0.1716,0.1641,0.1571,0.1505,0.1443,0.1385,0.1332,0.1284,0.124,0.1201,0.1167,0.1137,0.1111,0.1089,0.1071,0.1056,0.1044,0.1034,0.1027,0.1022,0.1019,0.1018,0.1017,0.1018],\n",
    "    [0.2866,0.2752,0.264,0.2532,0.2427,0.2326,0.2228,0.2134,0.2044,0.1958,0.1876,0.1798,0.1724,0.1653,0.1587,0.1524,0.1465,0.141,0.1359,0.1313,0.1271,0.1233,0.12,0.1171,0.1145,0.1124,0.1106,0.1091,0.1079,0.1069,0.1062,0.1057,0.1054,0.1052,0.1051,0.1051],\n",
    "    [0.2729,0.2632,0.2538,0.2446,0.2357,0.227,0.2187,0.2106,0.2028,0.1954,0.1882,0.1813,0.1748,0.1685,0.1626,0.1569,0.1516,0.1465,0.1418,0.1376,0.1336,0.1301,0.127,0.1242,0.1218,0.1197,0.1179,0.1164,0.1152,0.1143,0.1135,0.113,0.1126,0.1124,0.1122,0.1122],\n",
    "    [0.2548,0.2473,0.24,0.2329,0.226,0.2192,0.2126,0.2063,0.2001,0.1941,0.1883,0.1827,0.1773,0.1721,0.167,0.1622,0.1576,0.1532,0.149,0.1452,0.1416,0.1383,0.1354,0.1327,0.1303,0.1281,0.1262,0.1246,0.1232,0.122,0.121,0.1202,0.1195,0.119,0.1186,0.1183],\n",
    "    [0.2482,0.242,0.2359,0.2299,0.2241,0.2184,0.2128,0.2074,0.2021,0.1969,0.1919,0.187,0.1823,0.1777,0.1733,0.169,0.1649,0.1608,0.157,0.1535,0.1501,0.1471,0.1442,0.1415,0.1391,0.1369,0.135,0.1332,0.1316,0.1303,0.1291,0.128,0.1271,0.1263,0.1257,0.1252],\n",
    "    [0.2331,0.2278,0.2227,0.2176,0.2127,0.2078,0.2031,0.1984,0.1939,0.1895,0.1851,0.1809,0.1768,0.1728,0.1689,0.1652,0.1615,0.158,0.1547,0.1516,0.1486,0.1459,0.1433,0.1409,0.1387,0.1367,0.1349,0.1332,0.1317,0.1303,0.1291,0.128,0.127,0.1261,0.1254,0.1247],\n",
    "    [0.2313,0.2262,0.2212,0.2163,0.2115,0.2068,0.2022,0.1977,0.1932,0.1889,0.1847,0.1806,0.1766,0.1727,0.1689,0.1653,0.1617,0.1582,0.155,0.152,0.1491,0.1464,0.1438,0.1415,0.1393,0.1373,0.1354,0.1337,0.1322,0.1308,0.1296,0.1284,0.1274,0.1266,0.1258,0.1251],\n",
    "    [0.2302,0.2253,0.2206,0.216,0.2114,0.2069,0.2026,0.1983,0.1941,0.19,0.186,0.1821,0.1783,0.1745,0.1709,0.1674,0.164,0.1607,0.1576,0.1546,0.1518,0.1492,0.1467,0.1444,0.1423,0.1403,0.1384,0.1367,0.1351,0.1337,0.1324,0.1313,0.1302,0.1293,0.1285,0.1277],\n",
    "    [0.229,0.2249,0.2209,0.2169,0.213,0.2091,0.2054,0.2017,0.1981,0.1946,0.1911,0.1877,0.1845,0.1812,0.1781,0.175,0.172,0.1691,0.1664,0.1637,0.1611,0.1587,0.1563,0.1541,0.152,0.15,0.148,0.1462,0.1445,0.143,0.1415,0.1401,0.1388,0.1376,0.1365,0.1355],\n",
    "    [0.2289,0.2249,0.2209,0.2171,0.2133,0.2096,0.2059,0.2023,0.1988,0.1953,0.192,0.1887,0.1855,0.1823,0.1793,0.1763,0.1733,0.1705,0.1678,0.1651,0.1626,0.1602,0.1579,0.1557,0.1535,0.1515,0.1496,0.1478,0.1461,0.1445,0.143,0.1415,0.1402,0.139,0.1379,0.1368],\n",
    "    [0.2239,0.2207,0.2175,0.2143,0.2112,0.2082,0.2051,0.2022,0.1993,0.1964,0.1936,0.1909,0.1882,0.1855,0.1829,0.1804,0.1779,0.1755,0.1731,0.1708,0.1685,0.1663,0.1642,0.1622,0.1601,0.1582,0.1563,0.1545,0.1527,0.151,0.1494,0.1478,0.1464,0.1449,0.1436,0.1423]\n",
    "]\n",
    "df2 = pd.DataFrame(d, columns=c, index=i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.graph_objs as go\n",
    "\n",
    "fig = go.Figure(\n",
    "    data=[ go.Surface(\n",
    "        z=df2.values.tolist(),\n",
    "        y=df2.columns.values,\n",
    "        x=df2.index.astype(str).values.tolist()\n",
    "    )],\n",
    "    layout=dict(\n",
    "        title = 'Vol Surface', \n",
    "        autosize = True,\n",
    "        width = 900,\n",
    "        height = 700,\n",
    "        margin = dict(\n",
    "            l = 65,\n",
    "            r = 50,\n",
    "            b = 65,\n",
    "            t = 90\n",
    "        ),\n",
    "        scene = dict(\n",
    "            aspectratio = dict(\n",
    "                x = 1,\n",
    "                y = 1,\n",
    "                z = 0.667\n",
    "            )\n",
    "        )\n",
    "    ))\n",
    "\n",
    "go.FigureWidget(fig)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
